Music

Good morning all together. Welcome for computer graphics.

Today we will speak about head-mounted displays again. So, yeah, sorry for that we started with virtual reality.

Then we had the Unity demo on Monday. I hope that was interesting.

And today we will make the second part on virtual reality and in particular on head-mounted devices.

And in this part we will speak about the core component, I would say, of head-mounted displays.

And that's the tracking part. The tracking of the display is one of the major differences from normal computer graphics where we sit in front of a screen.

So, of course, in fact there are two things. The first one is the stereo rendering. We spoke about that last week.

And today we will speak about the tracking part. Yeah? So, how does that tracking work?

And in particular for head-mounted displays there's a very important aspect which is latency.

That's the time between moving the head and seeing the change of that head movement on the display.

And, of course, that's very closely related to tracking. But it also means that in our rendering pipeline for head-mounted displays we have to take this into account to keep that latency as low as possible.

And that's a core component of virtual reality rendering. And that's what we will speak about then in the second part.

But it's all related to that tracking, to the fact that with a head-mounted display when we move the head automatically the image adapts to that head movement so that we feel immersed into that virtual environment.

And, yeah, tracking is very important. And good tracking is very important for HMD rendering.

So, first of all it's very important that we have good reliable tracking. We will see normal sensors that measure position, for instance, generate things like noise.

So the measurement is rather unprecise and moves back and forward continuously. If you would use that information for rendering in a head-mounted display that would be horrible.

Because the world around you would always be shaking. That definitely would not be a good experience.

And another probably more subtle aspect I already said so is that the latency must be low. That means when we move the head it immediately or as fast as possible this movement should also be visible on the display.

Otherwise if there's a lag between head movement and seeing that on the screen it quickly results in discomfort and people feel sick very quickly.

Okay, so head tracking. Let's start with that part. How is the head tracked or the display in space?

So, first of all there are two components that we have to measure. The first one is the head orientation.

So when I rotate my head maybe the position, the world position doesn't change but the orientation of the device changes.

And this has a large impact on the rendering so it needs to be measured.

And it's something different from the position. So a GPS for instance as you know it from cars or from your mobile phones.

It just delivers you a 3D world position but it doesn't tell you the orientation of your mobile phone.

Some of you probably know from your mobile phone also for orientation you have a sensor, a compass and that's another degree of freedom in tracking that we have to follow.

So we have view direction or head orientation and head position.

And for simple VR setups where you sit on a chair for instance often the head position is not important at all so it's really only the head orientation that is to be tracked.

But if we are in an environment where we can move freely and walk around a little bit then also the head position is important.

So we need these two different parameters and of course the tracking must be fast which mainly means low latency.

So probably you know that on mobile phones such position sensors that might help for our purpose are already available.

And in fact that's not so long that such sensors are available.

Today we find that completely normal but this is a rather new development.

And probably the most well known position sensor is a GPS on every mobile phone available.

Not expensive anymore but I think you know that the precision of GPS is definitely not sufficient for a head mounted display.

It is something in the range of 10 meters maybe a little bit less but for instance it usually doesn't work at all indoors which is a killer criteria for virtual reality.

Furthermore it's very power consuming which is also can be an issue if we have a head mounted display that has no power cable or so.

There are also such displays and it just doesn't work for our purpose.

What can help is another sensor that you know maybe from your mobile phone if in Google Maps for instance you do not only see your position but also the direction you're looking at.

And this is measured by a compass or called magnetoscope.

In fact the compass does not only deliver north, south, east the orientation but it delivers a 3D vector that shows the magnetic field at the current point.

And of course this gives us orientation about the sensor not only in this direction but also up and down because it delivers the 3D direction.

But of course rotations around the magnetic field cannot be measured and in particular indoors again there is again the problem that there are many other sources of magnetic fields in particular if you're working with computers.

So often the magnetic field indoors mostly comes from electronic devices and other stuff.

So it doesn't show you the magnetic field of the earth but of that current position and it can change while you walk around in a room.

So absolutely or the absolute direction of a compass is not reliable but at least it works in a relative mode.

That means if your magnetic field changes or if this vector changes that's a hint that you rotated the mobile phone somehow.

And it can also give a hint into which direction.

And yeah so it is used but it requires thought to use it. We will also come back to that and also the magnetoscope is power consuming.